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(54) Universal remote control adapted to receive voice input 



(57) A universal remote control (20) is adapted to 
receive a voice input via a receiver (42). The voice input 
is received by the remote control (20) and is compared 
with a plurality of voice command templates that are 
stored in the memory (45) of the remote control. If the 
voice input matches one or more of the plurality of voice 
command templates, a valid voice input has been re- 
ceived by the remote control. Valid voice input may be 
a remote control command or keystroke data input as 
an entire word or as individual characters. In response 
to a valid voice input, the remote control (20) may trans- 
mit an operation or command code and/or alpha numer- 
ic symbol code corresponding to keystroke data to a 
consumer electronic device (22). 
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Description 

BACKGROUND OF THE INVENTION 

[0001] The present invention relates generally to uni- 
versal remote control devices that are adapted to control 
the operation of consumer electronic devices and, more 
particularly, to a universal remote control that is adapted 
to recognize voice input for use in transmitting opera- 
tional command codes and/or alphanumeric symbol 
codes representative of keystroke data to consumer 
electronic devices. 

[0002] Universal remote controls capable of control- 
ling the operation of consumer electronic devices are 
well known in the art. For example, U.S. Pat. No. 
5,959,751 to Darbee etal., describes a universal remote 
control capable of controlling the operation of a plurality 
of different types of consumer electronic devices from a 
plurality of different manufacturers. Such a universal re- 
mote control is intended to primarily control the function- 
al operations of the consumer electronic devices them- 
selves. For example, to control the volume and channel 
tuning in a television; play, rewind and stop in a VCR; 
etc. 

[0003] Currently, the home entertainment environ- 
ment is expanding to include consumer electronic de- 
vices that are intended to communicate with other elec- 
tronic devices via a network, such as the Internet, in sup- 
port of on-line banking, shopping, and the like. To sup- 
port this additional functionality, the consumer electronic 
devices typically require a user to enter keystroke data 
(i.e., alphanumeric and punctuation symbols collectively 
referred to hereinafter as "alphanumeric symbols.") Ex- 
amples of keystroke data include, but are not limited to, 
credit card information, short email messages, delivery 
addresses for purchased items, Web site U RLs and the 
like. 

[0004] For allowing users to transmit keystroke data 
to consumer electronic devices, it is known to provide a 
keyboard. By way of example, U.S. Pat. No. 4,964,075 
to Shaver et al., U.S. Pat. No. 5,659,665 to Whelpley 
and U.S. Pat. No. 5,874,939 to Galvin each describe 
wired keyboards that include voice recognition capabil- 
ities. Shaver and Whelpley provide voice recognition 
features that are intended to be retrofitted to existing 
products. Galvin provides voice recognition features by 
incorporating a voice recognition module with a stand- 
ard keyboard. While these keyboards are capable of 
transmitting keystroke data to the consumer electronic 
devices, keyboards suffer the disadvantage of being 
large, unwieldy, and more costly than most remote con- 
trols. 

[0005] It is also known in the art to use touch screens 
as a means by which keystroke data is entered into a 
hand held device. For example, products such as Inter- 
link's Freedom Writer® and products by IntuiTouch or 
the Palm Pilot family offer various forms of "graffiti" in 
which the outline of characters are traced on a touch 



sensitive surface, with either a finger tip or a stylus, 
thereby producing keystroke data on the touch sensitive 
surface. Universal Electronics Incorporated ("UEI") and 
Philips also market products that provide an LCD behind 

5 a touch sensitive surface which displays a mini-key- 
board allowing users to select and enter keystroke data 
with a stylus, i.e., UEI's Mosaic® and Philips' Pronto®. 
While touch screens allow users to enter keystroke data, 
they suffer various disadvantages affecting user accept - 

10 ability, including the need to enter multiple keystrokes 
for each character and the need for users to learn how 
to form the "graffiti" characters necessary to input key- 
stroke data. 

[0006] Yet another approach for entering keystroke 
is data into a hand held device uses a standard telephone 
style numeric keypad labeled with letters of the alpha- 
bet. For example, the user enters each character as a 
pair of keystrokes - the first keystroke indicating which 
three (orfour) character group is being selected and the 
20 second keystroke indicating which character within that 
group is desired. While this approach does provide a 
method for users to enter text data, using keypads in 
this manner is both time consuming and difficult. 

25 SUMMARY OF THE INVENTION 

[0007] To overcome the disadvantages noted above, 
the present invention is directed to a universal remote 
control that is capable of transmitting operational com- 
30 mand codes and/or keystroke data to consumer elec- 
tronic devices without requiring a separate keyboard, 
touch screen, or telephone style numeric keypad. More 
specifically, the universal remote control is adapted to 
recognize voice input for use in transmitting operational 
35 command codes and/or keystroke data to a consumer 
electronic device. In this manner, voice input may be 
used to cause the remote control to command the op- 
eration of consumer electronic devices while also allow- 
ing the user to transmit keystroke data, such as Web 
to site addresses, to the consumer electronic devices. 
[0008] In accordance with the described remote con- 
trol, a voice input is received by the remote control which 
is compared to a plurality of voice command templates 
that are stored in the memory of the remote control. If 
45 the voice input matches one or more of the plurality of 
voice command templates, a valid voice input has been 
received by the remote control. Valid voice input may be 
a remote control command and/or keystroke data, input 
as an entire word or as individual characters. In re- 
50 sponse to a valid voice input, the remote control may 
transmit a corresponding operational command code 
and/or keystroke data to a consumer electronic device. 
[0009] A better understanding of the objects, advan- 
tages, features, properties and relationships of the in- 
55 vention will be obtained from the following detailed de- 
scription and accompanying drawings which set forth an 
illustrative embodiment and which are indicative of the 
various ways in which the principles of the invention may 
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be employed. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[001 0] For a better understanding of the invention, ref- 
erence may be had to a preferred embodiment shown 
in the following drawings in which: 

Figure 1 shows an exemplary remote control for use 
in accordance with the subject invention; 
Figure 2 shows schematic diagrams of the remote 
control illustrated in Fig. 1 ; 
Figure 3 shows the remote control of Fig. 1 trans- 
mitting keystroke data to a consumer electronic de- 
vice in response to a character-by-character voice 
input; 

Figure 4 shows the remote control of Fig. 1 trans- 
mitting keystroke data to a consumer electronic de- 
vice in response to a word voice input; and 
Figures 5a-5d show flow chart diagrams of an ex- 
emplary method by which the remote control of Fig. 
1 can transmit command codes, including keystroke 
data, to a remotely controlled device. 

DETAILED DESCRIPTION 

[0011] Turning now to the figures, wherein like refer- 
ence numerals refer to like elements, there is illustrated 
a remote control 20 adapted to transmit command 
codes, including operational command codes and al- 
phanumeric symbol codes representative of keystroke 
data, to remotely controlled consumer electronic devic- 
es 22 in response to voice input. The ability to transmit 
keystroke data is particularly advantageous as it allows 
the remote control 20 to meet the need for users of key- 
boardiess, electronic devices to navigate and engage in 
electronic business transactions on the Internet. As will 
be understood by those of ordinary skill in the art, the 
remote control 20 transmits signals to the consumer 
electronic devices 22 using any suitable wired or wire- 
less transmission means using coding formats recog- 
nized by the consumer electronic devices 22. 
[001 2] As shown in Fig. 2, the remote control 20 pref- 
erably includes a microprocessor 30, memories 35 an 
45, and a digital signal processor 40. An arrangement 
of buttons 70 is coupled to the microprocessor 30 for, 
among other things, allowing the user 10 to command 
the operation of the remote control 20 in a conventional 
manner. The buttons 70 include number buttons 72, 
function buttons 74, and mode buttons 76 including a 
keyboard button 78, and a voice button 80. As will be 
described in greater detail hereinafter, the voice button 
80 is provided for placing the re mote control 20 in a voice 
activated state while the keyboard button 78 is provided 
for placing the remote control 20 in a keyboard mode. 
While described in the context of physical buttons on the 
remote control 20, the buttons can be implemented vir- 
tually using touch screens or the like. 



[001 3] The memories 35 and 45 may be comprised of 
any type of computer-readable media, such as ROM, 
RAM, SRAM, FLASH, EEPROM, or the like. Preferably 
these comprise non-volatile forms of memory such as 

5 ROM, Flash, or battery-backed SRAM such that pro- 
grammed and user entered data is not required to be 
reloaded after battery changes. Furthermore, the mem- 
ories 35 and 45 may take the form of a chip, a hard disk, 
a magnetic disk, and/or an optical disk. 

w [0014] For use in controlling the operation of the re- 
mote control 20, the memory 35 includes executable in- 
structions that are intended to control the operation of 
the microprocessor 30. In this manner, the microproc- 
essor 30 may be programmed to control the various 

'5 electronic components within the remote control 20, e. 
g., to control power, to cause thetransmission of signals, 
etc. For communicating with different types of consumer 
electronic devices from different manufacturers, the 
memory 35 also includes an operational command code 

20 library as well as an alphanumeric symbols library. The 
operational command code library is comprised of a plu- 
rality of operational command codes that may be trans- 
mitted from the remote control 20, in response to acti- 
vation of a button or a voice input signal 52, to control 

25 the operation of a consumer electronic device. The al- 
phanumeric symbols library contains alphanumeric 
symbol codes representative of keystroke data that are 
similarly transmitted to consumer electronic devices in 
response to activation of a button or a voice input signal. 

30 in connection with the stored command and alphanu- 
meric symbol codes, the memory 35 includes instruc- 
tions which the processor uses in connection with a 
transmission module 90 to cause the codes to be trans- 
mitted in a format recognized by identifiable consumer 

35 electronic devices. As will be appreciated by those of 
skill in the art, a user may enter data into the remote 
control 20 that serves to identify consumer electronic 
devices by type and manufacturer such that the remote 
control 20 is adapted to transmit the codes in the formats 

<o appropriate for such identified consumer electronic de- 
vices. 

[0015] Within the memory 45 are stored executable 
instructions that are intended to control the operation of 
the digital signal processor 40 to implement a Voice 

45 Recognition Engine (VRE) 50. The memory 45 also 
stores the voice command templates that are used to 
determine whether a voice input signal 52 that is provid- 
ed by the user 1 0 to the remote control 20 is a valid voice 
input signal. Valid voice input signals 52 are those that 

50 match one or more of the voice command templates 
stored in the memory 45. In this regard, the voice com- 
mand templates to which the voice input signals are 
compared may be words and/or spoken alphanumeric 
symbols. It will be appreciated that the two memories 

55 35 and 45 need not both be of the same type; advanta- 
geously memory 35 may be ROM while memory 45 may 
be Flash or other non-volatile read/write memory in or- 
der to allow the user of the remote to store new voice 
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phrases (templates) for recognition. Additionally, each 
of these memories may themselves comprise a mixture 
of types, for instance either or both memories may in- 
clude a small amount of RAM for use as transient, or 
temporary, storage during processing. 
[001 6] For controlling the operation of the remote con- 
trol itself or one or more consumer electronic devices, 
the user 1 0 may activate one or more keys on the remote 
control 20. Consumer electronic devices that are espe- 
cially adapted for remote control include TVs, VCRs, 
DVD players, thermostats, fans, entry systems, comput- 
ers, and similar home appliances. Local operations that 
may be included with the remote control 20, such as fa- 
vorite button setup, macro button setup, etc., are per- 
formed in accordance with the executable instructions 
that are used to control the operation of the remote con- 
trol 20. As noted previously, these executable instruc- 
tions may also be activated in response to voice input. 
[0017] For receiving the voice input, a receiver 42, 
such as a microphone, is provided. The receiver 42 is 
linked to the digital signal processor 40 which is coupled 
to the microprocessor 30. The voice input, which is ini- 
tially received as an analog signal, is converted to a dig- 
ital signal whereby it may be compared against the voice 
command templates stored in the memory 45. The A/D 
conversion of the voice signal may be performed in ac- 
cordance with known principles using an amplifier 44 
and filter 46 (for enhancing the voice input signal 52 by 
increasing the strength of the signal and removing any 
noise) in connection with a standard analog to digital 
converter 48. The digitized voice input signal is then 
processed by the voice recognition engine (VRE) 50, a 
set of programming instructions for the DSP 40 which 
compare the voice input signal with the voice command 
templates stored in the memory 45. 
[0018] For use in comparing the voice input signal 
with the voice command templates stored in the memory 
45, the voice recognition engine 50 may be provided 
with a comprehensive word recognition capability. In ac- 
cordance with a comprehensive word recognition capa- 
bility, the voice recognition engine 50 will parse whole 
words said by the user, for example, "www," "dot," "ya- 
hoo," "dot," "com," into individual alphanumeric sym- 
bols. Alternatively, the remote control may utilize a less 
expensive, low-level version of the voice recognition en- 
gine 50 that is only programmed to identify alphanumer- 
ic symbols. In this case, the recognition engine simply 
operates on the individual sounds said by the user, thus 
requiring the user to spell out each word one character 
at a time, for example, "w," "w," "w." "dot," "y," "a," "h " 
'V "o," "dot," "c," "o," "m." 

[0019] To access the voice recognition capabilities of 
the remote control 20, as shown in Figs. 5a - 5d, the 
user 1 0 activates the voice button 80 to place the remote 
control in a voice activated state. Once the voice button 
80 is activated or depressed, the microprocessor 30 
powers up the digital signal processor 40, activates the 
voice recognition engine 50, and initializes the receiver 



42. After the voice recognition engine 50 is activated and 
the receiver 42 is initialized, the voice recognition engine 
50 waits for a voice input signal 52. 
[0020] When not in the voice activated state, the re- 
mote control 20 preferably monitors the device mode 
status as set by the most recent user activation of one 
of the mode buttons 76, 78. If the mode is not "keyboard" 
(i.e., one of the device type mode buttons 76 was last 
activated) the remote control 20 is in a mode whereby 
the command buttons of the remote control are used to 
command the operation of consumer electronic devices 
22 in a conventional manner. If, however, the keyboard 
mode button 78 was the last mode button activated, the 
remote control is in a keyboard mode whereby activation 
of a command button is understood to be the equivalent 
of entering keystroke data. For example, in the keyboard 
mode, activation of a number button would not cause 
the transmission of an operational change channel com- 
mand to the consumer electronic devices as would be 
conventional. Rather, the activation of a number button 
would cause the remote control to transmit a formatted 
alphanumeric symbol code corresponding to the 
number button activated. This is particularly advanta- 
geous since it allows a security conscious user to input 
keystroke data (e.g., a credit card number) without hav- 
ing to speak it aloud. 

[0021] When the remote control 20 is in the voice ac- 
tivated state and the voice recognition engine 50 is wait- 
ing for a voice-input signal 52, the microprocessor 30 
also preferably checks to see if the voice button 80 has 
been deactivated. If the voice button 80 has been deac- 
tivated, the microprocessor 30 should issue a stop com- 
mand to the digital signal processor 40. Once the digital 
signal processor 40 acknowledges receipt of the stop 
command, the microprocessor 30 should power down 
the digital signal processor 40. The microprocessor 30 
should also check to see if the digital signal processor 
40 has timed out, i.e., if a specified period of time has 
passed without a response or voice-input signal 52 from 
a user 10. If the digital signal processor 40 has timed 
out, the microprocessor 30 should again issue a stop 
command to the digital signal processor. 
[0022] To ensure that the remote control 20 accurately 
receives voice input signals 52 from the user 10, the mi- 
croprocessor 30 may check for the existence of an error 
condition, e.g., insufficient amplitude for the voice-input 
signal 52, excessive background noise or the like. If an 
error condition exists, the microprocessor 30 should in- 
dicate the existence of an error condition to the user by 
providing a beep, a flashing LED, a LCD message or 
similar notification means. After indicating that an error 
condition exists, the microprocessor 30 should also is- 
sue a stop command to the digital signal processor. 
[0023] As noted previously, to determine whether a 
valid voice signal was received, the voice recognition 
engine 50 compares the digitized voice signal to the 
voice command templates stored in memory 45. If a 
match is found, then a valid voice signal has been pro- 
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vided to the remote control 20 and a corresponding com- 
mand code (e.g., operational command code and/or al- 
phanumeric symbol code corresponding to keystroke 
data) may be transmitted to the consumer electronic de- 
vice 22 using the coding format recognizable by the con- 5 
sumer electronic device 22/ 

[0024] Taking the case of a voice recognition engine 
having a comprehensive word recognition capability, the 
voice recognition engine 50 may determine whether a 
word or character has been provided. For example, the 10 
voice recognition engine 50 may first compare the digi- 
tized voice signal to voice command templates that are 
also in the form of complete words. If the digitized voice 
signal is recognized as, i.e., matches, a voice command 
template, then the voice signal is determined to be a val- w 
id voice signal and the voice recognition engine 50 
sends a signal to the microprocessor 30 indicating that 
a valid word has been received. 
[0025] If the digital input signal is not a complete word, 
the voice recognition engine 50 compares the digitized 20 
voice signal to voice command templates that are stored 
in the form of alphanumeric symbols. If the voice signal 
is recognized as, i.e., matches, a voice command tem- 
plate, then the digitized voice signal is valid. In response 
to the receipt of one or more valid alphanumeric voice 25 
commands, the voice recognition engine 50 sends a sig- 
nal to the microprocessor 30 indicating that a valid al- 
phanumeric symbol was received. 
[0026] If the microprocessor 30 receives a response 
indicating that a valid word was received, then the steps 
shown in Fig. 5c may be executed. In the example illus- 
trated, the microprocessor 30 issues a "read" command 
to the voice recognition engine 50. When the "read" 
command is received by the voice recognition engine 
50, the voice recognition engine 50 sends signals indic- 
ative of the parsed voice input (i.e., signals indicative of 
a string of alphanumeric symbols 64) to the microproc- 
essor 30. The microprocessor uses the signal from the 
voice recognition engine 50 to access the memory 34 
to look-up the information/instructions needed to cause 
the transmission module 90 to send an alphanumeric 
symbol code corresponding to each alphanumeric sym- 
bol in the voice input in the appropriate format for the 
intended consumer electronic device 22. Once all of the 
alphanumeric symbols have been transmitted to the 
consumer electronic device 22, the microprocessor 30 
sends an acknowledgement to the voice recognition en- 
gine 50. In this manner, if the user speaks the words 
"www," "dot," "yahoo," "dot," "com," the remote control 
20 will transmit the equivalent of each alphanumeric 
symbol included in the word sequence to the consumer 
electronic device 22 in the format recognized by the con- 
sumer electronic device 22. 

[0027] If the microprocessor 30 receives a response 
indicating that a valid alphanumeric symbol was re- 
ceived, then the steps shown in Fig. 5d may be execut- 
ed. In the illustrated example, the microprocessor 30 is- 
sues a "read" command to the voice recognition engine 
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50. In response to the "read" command, the voice rec- 
ognition engine 50 sends a signal indicative of the al- 
phanumeric voice input to the microprocessor 30 and 
the microprocessor 30 uses the signal to cause the 
transmission module to send the formatted alphanumer- 
ic symbol code corresponding to the voice input to the 
intended consumer electronic device 22. Once the al- 
phanumeric symbol code is sent, the microprocessor 30 
will send an acknowledgement to the voice recognition 
engine 50. This is repeated for each alphanumeric sym- 
bol the user inputs by voice. In this manner, if the user 
speaks the alphanumeric sounds "w," "w," "w," "dot," "y," 
"a," "h," M o/"o," "dot, M "c, M "o, B "m, n the remote control 20 
will transmit the equivalent of each of the alphanumeric 
symbols included in the sequence to the consumer elec- 
tronic device 22 in the format recognized by the con- 
sumer electronic device 22. 

[0028] Still further, the voice recognition engine can 
use the spoken words and/or alphanumeric sounds to 
cause the remote control 10 to transmit operational 
command codes to the consumer electronic devices. 
The user 1 0 can utter certain pre-programmed phrases 
which will cause the voice recognition engine to direct 
the microprocessor to initiate the transmission of a cor- 
responding operational command code to the consumer 
electronic devices in formats appropriate for the con- 
sumer electronic devices. For example, in response to 
the user uttering the letters "E " "S," "P," and "N," the 
remote control 20 can be adapted to transmit one or 
more operational command codes to an identified cable 
box that causes the cable box to tune the attached tel- 
evision to the channel carrying "ESPN." 
[0029] The order of all steps disclosed in the figures 
and discussed above has been provided for exemplary 
purposes only. Therefore, it should be understood by 
those skilled in the art that these steps may be rear- 
ranged and altered without departing from the spirit of 
the present invention. In addition, it is to be understood 
that all patents discussed in this document are to be in- 
corporated herein by reference in their entirety. 
[0030] While specific embodiments of the present in- 
vention have been described in detail, it will be appre- 
ciated by those skilled in the art that various modifica- 
tions and alternatives to those details could be devel- 
oped in light of the overall teachings of the disclosure. 
For example, the processes described with respect to 
computer executable instructions can be performed in 
hardware or software without departing from the spirit 
of the invention. Accordingly, the particular arrangement 
disclosed is meant to be illustrative only and not limiting 
as to the scope of the invention which is to be given the 
full breadth of the appended claims and any equivalents 
thereof. 
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being activatable to enter button input, a micro- 
phone for receiving voice input, and a memory in 
which is storedapluralityofcommandcodes for use 
in transmitting signals for commanding the opera- 
tion of consumer electronic devices and for use in 
transmitting signals representative of alphanumeric 
symbols, a method for transmitting signals to a con- 
sumer electronic device representative of an input 
to the remote control comprising: 

determining if the input is a voice input or a but- 
ton input; 

if the input is a voice input, selecting at least 
one command code from the plurality of com- 
mand codes stored in the memory as a function 
of the voice input, the command codes being 
selected from the command codes for use in 
transmitting signals representative of alphanu- 
meric symbols; 

if the input is button input, selecting at least o ne 
command code from the plurality of command 
codes stored in the memory as a function of the 
button or buttons activated; and 
using the selected command code to transmit 
a signal whereby the transmitted signal is rep- 
resentative of the voice input or the button in- 
put. 

The method as recited in claim 1 , wherein the plu- 
rality of buttons includes a keyboard button for plac- 
ing the remote control in a keyboard mode and, if 
the remote control is placed in the keyboard mode, 
the step of selecting at least one command code 
from the plurality of command codes stored in the 
memory as a function of the button or buttons acti- 
vated comprises selecting the command code from 
the command codes for use in transmitting signals 
representative of alphanumeric symbols. 

The method as recited in claim 1 , wherein the plu- 
rality of buttons includes a voice button for placing 
the remote control in a mode for receiving voice in- 
put and the step of selecting at least one command 
code from the plurality of command codes stored in 
the memory as a function of the voice input is con- 
ditioned upon the state of the voice button. 

A readable media having instructions for command- 
ing the operation of a remote control having a plu- 
rality of buttons each being activatable to enter but- 
ton input, a microphone for receiving voice input, 
and a memory in which is stored a plurality of com- 
mand codes for use in transmitting signals for com- 
manding the operation of consumer electronic de- 
vices and for use in transmitting signals represent- 
ative of alphanumeric symbols, the instructions per- 
forming the steps comprising: 



10 



15 



20 



25 



30 6. 



35 



determining if an input to the remote control is 
a voice input or a button input; 
if the input is a voice input, selecting at least 
one command code from the plurality of com- 
mand codes stored in the memory as a function 
of the voice input, the command codes being 
selected from the command codes for use in 
transmitting signals representative of alphanu- 
meric symbols; 

if the input is button input, selecting at least one 
command code from the plurality of command 
codes stored in the memory as a function of the 
button or buttons activated; and 
using the selected command code to transmit 
a signal whereby the transmitted signal is rep- 
resentative of the voice input or the button in- 
put. 

The readable media as recited in claim 4, wherein 
the plurality of buttons includes a keyboard button 
for placing the remote control in a keyboard mode 
and, if the remote control is placed in the keyboard 
mode, the step of selecting at least one command 
code from the plurality of command codes stored in 
the memory as a function of the button or buttons 
activated comprises selecting the command code 
from the command codes for use in transmitting sig- 
nals representative of alphanumeric symbols. 

The readable media as recited in claim 1 , wherein 
the plurality of buttons includes a voice button for 
placing the remote control in a mode for receiving 
voice input and the step of selecting at least one 
command code from the plurality of command 
codes stored in the memory as a function of the 
voice input is conditioned upon the state of the voice 
button. 
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